Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[cloud provider] Add AWS EC2 instance id semantic convention #600

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

mx-psi
Copy link
Member

@mx-psi mx-psi commented Dec 12, 2023

Changes

Add semantic convention for AWS EC2 instance id. This is similar to #15, and is inspired by #576, to ensure that the EC2 instance id can be correctly identified despite ambiguity in the host.id convention meaning.

Note: if the PR is touching an area that is not listed in the existing areas, or the area does not have sufficient domain experts coverage, the PR might be tagged as experts needed and move slowly until experts are identified.

Merge requirement checklist

@mx-psi mx-psi marked this pull request as ready for review December 12, 2023 11:39
@mx-psi mx-psi requested review from a team December 12, 2023 11:39
@mx-psi mx-psi mentioned this pull request Dec 12, 2023
3 tasks
@mx-psi mx-psi requested a review from AlexanderWert December 12, 2023 11:41
model/resource/cloud_provider/aws/ec2.yaml Outdated Show resolved Hide resolved
Comment on lines +14 to +15
When both `host.id` and `aws.ec2.instance.id` are present,
they SHOULD be equal.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be phrased as follows to avoid duplication?

Suggested change
When both `host.id` and `aws.ec2.instance.id` are present,
they SHOULD be equal.
When the value in `host.id` is the same value as the `instance-id`
returned by the metadata endpoint, only `host.id` should be set.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per #576 (comment) I believe this is not what we do in similar cases such as with service.instance.id and k8s.pod.name. I think the situation is very similar here and we should do the same.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arminru PTAL!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It certainly makes the logic for both setting and reading the attribute easier. The duplicated string should in transport be taken care of by compression and also backends storing it would be able to deduplicate it as well.

I don't think we ever made a proper policy decision on whether to favor simplicity (for both instrumentation and querying) or deduplication but I believe either is a viable approach to take.

@open-telemetry/specs-semconv-approvers WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we ever made a proper policy decision on whether to favor simplicity (for both instrumentation and querying) or deduplication but I believe either is a viable approach to take.

fwiw, we abandoned de-duplication in network.peer.address etc (328a2c6) because it made it hard to know whether an attribute is missing because it was a duplicate or because the particular instrumentation doesn't capture it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this moves in a direction I'm hoping to push resource-detection -

That is a division of responsibilities. An AWS detector would be responsible for this attribute but NOT host.id.

Instead a Host Detector would generically detect host attributes as best as possible across known host-lookup mechanisms.

I'm writing up thoughts on this now, but I'd be in favor of this PR

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm writing up thoughts on this now, but I'd be in favor of this PR

@jsuereth Do you have any updates on this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm supportive of the addition as originally proposed by @mx-psi

@mx-psi mx-psi requested a review from pyohannes December 12, 2023 12:09
Copy link

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label Jan 25, 2024
Copy link

github-actions bot commented Feb 2, 2024

Closed as inactive. Feel free to reopen if this PR is still being worked on.

@mx-psi
Copy link
Member Author

mx-psi commented Feb 2, 2024

Reopening, waiting on @arminru's input above

@arminru arminru removed the Stale label Feb 2, 2024
@joaopgrassi
Copy link
Member

Hi @mx-psi !

We changed how the CHANGELOG.md is managed. Please take a look at https://github.com/open-telemetry/semantic-conventions/blob/main/CONTRIBUTING.md#adding-a-changelog-entry to see what needs to be done. Sorry for the disruption.

@mx-psi
Copy link
Member Author

mx-psi commented Feb 5, 2024

Thanks @joaopgrassi, addressed. It looks like there is some sort of issue with the changelog check though that is (AFAICT) unrelated to this PR:

Error: flag needs an argument: --version

@mx-psi
Copy link
Member Author

mx-psi commented Feb 13, 2024

Filed #739 for the general issue I am trying to solve by adding this.

Copy link

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added Stale and removed Stale labels Mar 16, 2024
brief: >
Resources used by Amazon Elastic Compute Cloud (Amazon EC2).
attributes:
- id: instance.id
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mx-psi Can you move the definition of this attribute to the registry and use a reference here? (see https://github.com/open-telemetry/semantic-conventions/pull/434/files for an example)

Copy link

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label Apr 12, 2024
Copy link

Closed as inactive. Feel free to reopen if this PR is still being worked on.

@github-actions github-actions bot closed this Apr 19, 2024
@mx-psi mx-psi reopened this Apr 19, 2024
@mx-psi mx-psi removed the Stale label Apr 19, 2024
Copy link

github-actions bot commented May 5, 2024

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label May 5, 2024
@trisch-me
Copy link
Contributor

@mx-psi could you please update this PR to have all attributes defined in the registry and autogenerated and use only refs outside of the registry?

@lmolkova
Copy link
Contributor

lmolkova commented May 7, 2024

I'd like to get a wider discussion on different attributes we have for host.id, service.instance.id, aws.ec2.instance.id and any others. It seems to be in the scope of Resources and Entities WG.

I think we should either wait for the Resource project to start and get consensus in that WG or at least discuss it in the SemConv SIG meeting.

@github-actions github-actions bot removed the Stale label May 8, 2024
@mx-psi
Copy link
Member Author

mx-psi commented May 9, 2024

I'd like to get a wider discussion on different attributes we have for host.id, service.instance.id, aws.ec2.instance.id and any others. It seems to be in the scope of Resources and Entities WG.

I think we should either wait for the Resource project to start and get consensus in that WG or at least discuss it in the SemConv SIG meeting.

Should we close this PR then for now? Not sure what the implications are for this PR

@lmolkova
Copy link
Contributor

lmolkova commented May 9, 2024

Should we close this PR then for now? Not sure what the implications are for this PR

Please don't close it :)

I'm suggesting to join Semantic Conventions meeting on Mon 8am to discuss how we should treat all the instance ids.
Or an entities meeting on every other Thu 8am (happening right now).

The meetings details are available in https://github.com/open-telemetry/community

@mx-psi
Copy link
Member Author

mx-psi commented May 10, 2024

It's usually hard for me to join this meeting, but I may be able to join on Monday.

@mx-psi
Copy link
Member Author

mx-psi commented May 16, 2024

Based on #576 (comment), I am going to hold off from further making changes on this PR until the Entities SIG gets up to speed

Copy link

github-actions bot commented Jun 1, 2024

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label Jun 1, 2024
@arminru arminru removed the Stale label Jun 5, 2024
@joaopgrassi joaopgrassi added the never stale PRs marked with this label will be never staled and automatically closed label Jun 13, 2024
@@ -0,0 +1,17 @@
groups:
- id: aws.ec2
prefix: aws.ec2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please stop using prefix.

@jsuereth
Copy link
Contributor

Given the status of Entities WG and updates to semconv, this PR can now make progress and be submitted.

It has the following:

  1. A well defined resource "name" / type that also is the "namespace" for attributes.
  2. A well defined notion of "identity" i.e. identifying attributes.

Thanks and sorry for the long delay!

@mx-psi
Copy link
Member Author

mx-psi commented Oct 16, 2024

Given the status of Entities WG and updates to semconv, this PR can now make progress and be submitted.

It has the following:

  1. A well defined resource "name" / type that also is the "namespace" for attributes.
  2. A well defined notion of "identity" i.e. identifying attributes.

Thanks and sorry for the long delay!

Thank you for the update! I will update this PR then (but it may take me some time to get back to it)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:system never stale PRs marked with this label will be never staled and automatically closed
Projects
Development

Successfully merging this pull request may close these issues.

8 participants